Let a custom profile inherit models with extends - #693
Merged
Conversation
A profile that differs from an existing one in a single model had to repeat all ten entries, so the copies drift and the actual difference is hard to see. A profile may now name a base with `extends` and state only what it changes. Resolution is a deep merge, so a profile can override one key of one model and inherit the rest of that model rather than replacing the whole entry. Chains are allowed. An unknown base and a cycle each raise, rather than failing later with a confusing model path; the cycle names the chain in the order it was followed, which is why the visited profiles are carried as a tuple rather than a set.
Gives extends a caller in the shipped config and a place for custom models to be swapped in one at a time. It currently states no models of its own, so it resolves to exactly what grobid_crf resolves to, which is what the test asserts - a model it later overrides will show up there as a difference. The processors block is repeated rather than inherited: extends applies to sequence_model_profiles, and profiles have no equivalent.
ScienceBeam Parser EvaluationOverall (60 docs across 6 corpora)grobid 0.9.0-crf (default): 60 docs | sciencebeam-parser:main-487b2b93-20260817.1008 (grobid_crf): 60 docs | sciencebeam-parser:pr-693-8fd9d731-20260817.1050 (grobid_custom_hybrid): 60 docs
biorxiv (10 docs)grobid 0.9.0-crf (default): 10 docs | sciencebeam-parser:main-487b2b93-20260817.1008 (grobid_crf): 10 docs | sciencebeam-parser:pr-693-8fd9d731-20260817.1050 (grobid_custom_hybrid): 10 docs
ore (10 docs)grobid 0.9.0-crf (default): 10 docs | sciencebeam-parser:main-487b2b93-20260817.1008 (grobid_crf): 10 docs | sciencebeam-parser:pr-693-8fd9d731-20260817.1050 (grobid_custom_hybrid): 10 docs
pkp (10 docs)grobid 0.9.0-crf (default): 10 docs | sciencebeam-parser:main-487b2b93-20260817.1008 (grobid_crf): 10 docs | sciencebeam-parser:pr-693-8fd9d731-20260817.1050 (grobid_custom_hybrid): 10 docs
scielo_br (10 docs)grobid 0.9.0-crf (default): 10 docs | sciencebeam-parser:main-487b2b93-20260817.1008 (grobid_crf): 10 docs | sciencebeam-parser:pr-693-8fd9d731-20260817.1050 (grobid_custom_hybrid): 10 docs
scielo_mx (10 docs)grobid 0.9.0-crf (default): 10 docs | sciencebeam-parser:main-487b2b93-20260817.1008 (grobid_crf): 10 docs | sciencebeam-parser:pr-693-8fd9d731-20260817.1050 (grobid_custom_hybrid): 10 docs
scielo_preprints-jats (10 docs)grobid 0.9.0-crf (default): 10 docs | sciencebeam-parser:main-487b2b93-20260817.1008 (grobid_crf): 10 docs | sciencebeam-parser:pr-693-8fd9d731-20260817.1050 (grobid_custom_hybrid): 10 docs
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
part of https://github.com/eLifePathways/ScienceBeam2.0/issues/113
A profile that differs from an existing one in a single model had to repeat
all ten model entries. Those copies drift from the base as it moves, and the
review diff for a new profile shows ten paths when only one of them is the
point.
A profile can now name a base with
extendsand state only what it changes,so putting a retrained model in front of the benchmark is a two-line profile
plus the one model that differs. Resolution is a deep merge, so overriding a
single key of a single model keeps the rest of that model rather than
replacing the whole entry. Chains work. An unknown base or a cycle raises
where the config is resolved, naming the chain it followed, rather than
surfacing later as a puzzling model path.
Adds
grobid_custom_hybridalongside it, extendinggrobid_crf_0_9_0andstating nothing of its own - the place for the retrained reference-segmenter
and citation models to be swapped in one at a time. It resolves to exactly
what
grobid_crfresolves to today, asserted against the shipped config, sothe first real override shows up there as a difference rather than as a
wholesale replacement.
Nothing changes for the existing profiles.